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Regression analysis, in statistic a modelling, is a set of statical processes that 
can be used to estimate the relationship between a dependent variable, 
commonly known as the outcome or response, and more independent 
variables generally called predictors of covariant. On the other hand, 
autoregression, which is based on regression equations, is a sequential model 
that uses time to predict the next step data from the previous step. Given the 
importance of accurate modelling and reliable predictions. in this paper we 
have analyzed the most popular methods used for data prediction. Nonlinear 
autoregressive methods were introduced, and then the machine deep learning 
approach was used to apply prediction based on a selected input data set. 
The mean square error was calculated for various artificial neural networks 
architecture to reach the optimal architecture, which minimized the error. 
Different artificial neural network (ANN) architectures were trained, tested, 
and validated using various regressive models, a recommendation was raised 


according to the obtained and analyzed experimental results. It was shown 
that using the concepts of machine deep learning will enhance the response 
of the prediction model. 
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1. INTRODUCTION 

In statistics, random data cannot be fully evaluated by visual inspection. While there are many types 
of regression analysis, at their core all methods examine the effects of one or more independent variables on 
a dependent variable. The regression analysis model (RAM) is the simple and effective model used to predict 
an output based on a selected input data set [1]-[3]. The implementation of RAM leads to a calculated 
regression coefficient, which was used in forming the predicted output equations as shown in Figure 
1 [4], [5]. 
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Figure 1. RAM equation 
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The regression means square error tells us how close the line is to a set of points. It was called the 
mean square error as it’s used in finding the average set of errors. Squaring was needed to eliminate the 
negative signs in MSE. The lower the MSE the better the forecast. The regression means square error (MSE) 
between the calculated and the target outputs may be high, causing a poor effective prediction. thus, the 
machine deep learning approach can be used to minimize MSE and enhance the prediction process accuracy 
[6], [7]. The machine deep learning (MDL) approach is a process of using an artificial neural network (ANN) 
[8]-[10] as a prediction tool. MDL can be treated using complicated ANN models, more calculations and big 
input data set, as shown in Figure 2 [11]-[14]. 
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Figure 2. ANN using MDL 


The minimum description length principle (MDL) is based on the concept of describing the data in 
full using fewer symbols. MDL, which was originated from algorithmic coding theory in computer science, 
regards both the model and the data as codes. The idea of MDL was to reduce redundancies by compression 
algorithms [15], [16]. Therefore, we can uncover irregular data and enhance the MSE results. In MDL the big 
input data set can be divided into training, testing, and validating data sets as shown in Figure 3. Artificial 
Neural Networks (ANNs) are computational algorithms s. Originally, it was intended to simulate biological 
systems composed of “neurons”. ANN is capable of machine learning and pattern recognition based on the 
model of human neurons. The interconnected neuron can compute values from inputs. ANNs are information 
process techniques that can be used for classifications and regression of continuous target attributes [17], 
[18]. This work will have a complicated architecture with a big number of neurons in the hidden layer, the 
selected number of training iterations must lead to MSE minimization [19]-[21]. 
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Figure 3. ANN deep learning 
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A nonlinear autoregressive model (NARM) can be built based on (1) and (2). In (1) the predicted 
output can be treated as a function of previous inputs and outputs as shown in Figure 4, while in (2) the 


predicted output is treated as a function of previous inputs only. We can see that there’s a feedback 
connection between the input and the output to get previous input [22], [23]. 


y(t) = F(x(t), x(t — 1), x(t — 2), ... x(t — d), y(t — 2),..., y(t — d)) (1) 


y(t) = F(x(t), x(t — 1), x(t — 2), ... x(t — d)) (2) 


Hidden layer with delays 


| Feedback connection 


Figure 4. NARM based on equation 1 


2. RESEARCH METHOD 

A pollution input data set was selected [2]-[5]. This data set contains a 2D matrix with 8 rows and 
508 columns, each column represents the values of PM (temperature, relative humidity, carbon monoxide, 
sulfur dioxide, nitrogen dioxide, hydrocarbons, ozone) [24], [25], The target data is a 2D matrix with 3 rows 
and 508 column, each row represents the values of the pollution negative effects (total mortality, respiratory 
mortality, cardiovascular mortality), Figure 5 shows a sample of the input data set. The input data set was 
divided into 3 parts: training, testing and validating data, the various ANN architectures were selected to be 
trained and used as a prediction tool. Figure 6 shows the steps required to test each selected ANN. 
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Figure 5. Samples of the input data set 
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Figure 6. Training process 
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The selected input data set and targets were treated using the regression model presented by Matlab 
code, Figure 7 shows the obtained regression coefficients, using these coefficients the targets formulas were 
calculated and the value of MSE was around 5.37425, as shown in Figure 8. To reduce MSE value NARM 
was built and tested. First, we used (1), where the output is a function of the previous inputs and outputs, and 
Figure 9 shows the architecture of the used ANN. Here we changed the number of neurons in the hidden 
layer and the number of delays, Figure 10 shows one of the responses of the executed model. The previous 
model was treated using a variant number of neurons in the hidden layer and various numbers of delays, 


Table 1 shows the obtained values for MSE 


Coeffecients 
t1 t2 t3 
220.3519 15.3448 134.3318 
-0.8580 -0.1126 -0.6681 
-0.2183 -0.0158 -0.1815 
1.3875 0.0779 1.0582 
2.4728 -0.0530 3.4070 
-0.9017 -0.0227 -0.6651 
0.1590 0.0216 0.0169 
0.7085 0.0199 0.2887 
0.0725 0.0183 0.0169 


Figure 7. Calculated regression coefficients 
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Figure 8. Regression errors between the targets and calculated outputs 
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Figure 9. NARM based on (1) 
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Figure 10. Model-based on (1) response 


Table 1. Obtained results of executing model 1 


Neuron /delay 8 4 2 
128 0.004428 0.216104 7.07005*e-14 
64 1.30904 0.144605 31.89164 
32 1.80656 23.46777 14.88772 
16 4.29606 12.18773 29.02915 
8 15.30561 16.25447 29.33317 
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From Table 1 we can see that increasing the number of neurons keeping the delay equal to 2 will 
enhance the model response by decreasing the MSE value, this is also shown in Table 2. Another model was 
built and tested based on (2), where the outputs were calculated as functions of the previous inputs, Figure 11 
shows the basic architecture of the used model. Here we also changed the number of neurons in the hidden 
layer and the number of delays, Figure 12 shows one of the responses of the executed model. This model was 
treated using a variant number of neurons in the hidden anandrousarious numbers of delays, Table 3 shows 
the obtained values for MSE. From Table 3 we can see that increasing the number of neurons in the hidden 
layer with a bigger number of delays will enhance the model response by decreasing MSE values.Examining 
the obtained result using the regression model, model 1 and model 2 we can see the following facts: i) both 
NAR models enhance the regression performance compared with the regression model; ii) applying MDL by 
increasing the number of neurons in the ANN hidden layer will enhance the model response by decreasing 
MSE values; iii) it is better to use model 2 for prediction by increasing the number of neurons in the ANN 
hidden layer and keeping the number of delays small; and iv) model 1 has a better performance compared 
with mode 2. 


Table 2. Effects of an increasing number of neurons keeping delay equal to 2 


No of Neuron MSE 
128 7.07005*e-14 
256 1.43335*e-14 
512 1.77292*e-26 
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Figure 11. NARM based on (2) 
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Figure 12. Model-based on (2) response 


Table 3. Obtained results of executing model 1 


Neuron /delay 8 4 2 
256 7.32080e-4 2.88122 e-2 1.49598 
128 1.0268 le-2 1.051855 37.54507 
64 1.85082 13.86173 1.41662 
32 3.2138 1le-3 21.91815 11.36876 
16 53.68 127 19.18600 39.52439 
8 57.53899 53.65321 38.15894 
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4. CONCLUSION 

A pollution data set was selected, the various target prediction models were built and tested, and the 
experimental results were examined and analyzed. It was shown that using the concepts of machine deep 
learning will enhance the response of the prediction model. Based on the obtained experimental results we 
recommend using either model | or model 2 for target prediction, increasing the number of neurons in model 
1 and keeping the number of delays small will lead to better performance by minimizing MSE values. 
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